Using paired comparison matrices to estimate parameters of the partial credit Rasch measurement model for rater-mediated assessments.
نویسندگان
چکیده
The purpose of this paper is to describe a technique for estimating the parameters of a Rasch model that accommodates ordered categories and rater severity. The technique builds on the conditional pairwise algorithm described by Choppin (1968, 1985) and represents an extension of a conditional algorithm described by Garner and Engelhard (2000, 2002) in which parameters appear as the eigenvector of a matrix derived from paired comparisons. The algorithm is used successfully to recover parameters from a simulated data set. No one has previously described such an extension of the pairwise algorithm to a Rasch model that includes both ordered categories and rater effects. The paired comparisons technique has importance for several reasons: it relies on the separability of parameters that is true only for the Rasch measurement model; it works in the presence of missing data; it makes transparent the connectivity needed for parameter estimation; and it is very simple. The technique also shares the mathematical framework of a very popular technique in the social sciences called the Analytic Hierarchy Process (Saaty, 1996).
منابع مشابه
Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model
In this study, the researcher used the many-facet Rasch measurement model (MFRM) to detect two pervasive rater errors among peer-assessors rating EFL essays. The researcher also compared the ratings of peer-assessors to those of teacher assessors to gain a clearer understanding of the ratings of peer-assessors. To that end, the researcher used a fully crossed design in which all peer-assessors ...
متن کاملRasch analysis of rank-ordered data.
Theoretical and practical aspects of several methods for the construction of linear measures from rank-ordered data are presented. The final partial-rankings of 356 professional golfers participating in 47 stroke-play tournaments are used for illustration. The methods include decomposing the rankings into independent paired comparisons without ties, into dependent paired comparisons without tie...
متن کاملApproximate measurement invariance in cross-classified rater-mediated assessments
An important assumption underlying meaningful comparisons of scores in rater-mediated assessments is that measurement is commensurate across raters. When raters differentially apply the standards established by an instrument, scores from different raters are on fundamentally different scales and no longer preserve a common meaning and basis for comparison. In this study, we developed a method t...
متن کاملThe impact of training on rater variability
In the five years, 1993 to 1998, total Commonwealth Government spending on education fell from 4.9 to 4.4 per cent of Gross Domestic Product. Australian universities have responded to this changed funding environment though the increased use of casual teaching staff. The aim of this study was to develop, implement and evaluate a short, cost-effective training package designed to improve the rat...
متن کاملInvestigating the Effect of Self-, Peer-, and Teacher Assessment in Second Language Writing over Time: A Multifaceted Rasch Approach
This study investigated the accuracy of scores assigned by self-, peer-, and teacher assessors over time. Thirty-three English majors who were taking paragraph development course at Vali-e-Asr University of Rafsanjan and two instructors who had been teaching essay writing for at least two years at university, participated in the study. After receiving instructions on paragraph development, part...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Journal of applied measurement
دوره 10 1 شماره
صفحات -
تاریخ انتشار 2009